Method and decoder for composing a scene

ABSTRACT

The present invention relates to a method for composing a scene containing a plurality of objects, an object comprising chrominance and luminance components, a chrominance value being associated with a set of at least two luminance values, wherein said method comprises a step of blending a first object with a second object resulting in a blended object, said step comprising the sub-steps of:—generating a luminance component of the blended object from the corresponding luminance components of the first and second objects and from a first composition function, and—generating a chrominance component of the blended object from the corresponding chrominance components of the first and second object and from a second composition function, the second composition function depending on a set of associated values of the first composition function.

FIELD OF THE INVENTION

The present invention relates to a method for composing a scenecontaining a plurality of objects, an object comprising chrominance andluminance components, a chrominance value being associated with a set ofat least two luminance values, said method comprising a step of blendinga first object with a second object resulting in a blended object.

The invention also relates to a decoder, said decoder implementing saidmethod. Such a method may be used in, for example, a MPEG-4 videocommunication system.

BACKGROUND OF THE INVENTION

A visual scene can be composed of objects of different type such asnatural video, two-dimensional, three-dimensional synthetic images,graphics or text.

A video communication system, like for example a mobiletelecommunication system, typically comprises a first mobile phone and asecond mobile phone, including an encoder and a decoder, respectively.In order to transmit a visual scene from the first mobile phone to thesecond mobile phone, the different objects are encoded by the firstmobile phone and transmitted independently to the second mobile phone.At the decoder side the scene is recomposed by decoding the differentobjects and then by rendering the decoded objects. The rendering usuallycomprises a geometric transformation step able to map the object on thescene and a step of blending the mapped objects together, resulting in afinal scene.

An object typically comprises chrominance U and V, and luminance Ycomponents forming the color of the object. During the blending step ofa current object N with an object M, a transparency value Nalpha can beassociated to the object N, said transparency value influencing thechrominance and luminance values Pyuv of a blending object P such that:Pyuv=[(255−Nalpha)*Myuv+Nalpha_(—) y*Nyuv]/255

The transparency of a pixel of a blended object is obtained according tothe shape of said pixel. The transparency permits to determine whichobject is behind or in front of another object in the scene. The shapeis associated with a pixel of a frame of a scene. It permits to know ifa pixel is outside or inside an object. If it is outside, the shape hasa null value, otherwise it has a value equal to 1.

For a pixel having coordinates (x,y), the shape value 0 or 1corresponding to these coordinates is taken and attributed to theluminance of this pixel. In the case where the luminance values thecrosses), the shape used for the chrominance value U or V(x,y) is theshape of the pixel of coordinates (2x−1, 2y−1), i.e. the top right pixelof a square of 2×2 pixels associated with the chrominance value.

One inconvenient of this solution is that the chrominance of the blendedobject is false. This results in a bad visual quality for objects havingan arbitrary shape. This observation is illustrated in the example of anobject constituted of the letter “e” and represented by stripes in FIG.2 a to 2 c. FIG. 2 a is the original object, FIG. 2 c is the result ofthe composition and FIG. 2 b explains how the result is obtained. Eachsquare represents a pixel. If one takes the pixel whose coordinates are(4,2), the pixel whose shape will be used for the chrominance componentis the pixel (3,1) as shown in FIG. 2 b. The value of this shape is 0,that is to say it is outside the letter “e”, and the chrominance valuewill not be taken into account, as shown in FIG. 2 c where the pixelvalue is represented by dots. The result is false because thechrominance value should have been taken into account and the resultshould have been the same as in FIG. 2 a.

SUMMARY OF THE INVENTION

Accordingly, it is an object of the invention to provide a method and adecoder for composing a scene containing a plurality of objects, anobject comprising chrominance and luminance components, a chrominancevalue being associated with a set of at least two luminance values, saidmethod comprising a step of blending a first object with a second objectresulting in a blended object, which gives a better visual quality ofsaid blended object.

To this end, there is provided a method wherein the blending stepcomprising the sub-steps of:

-   -   generating a luminance component of the blended object from the        corresponding luminance components of the first and second        objects and from a first composition function, and    -   generating a chrominance component of the blended object from        the corresponding chrominance components of the first and second        object and from a second composition function, the second        composition function depending on a set of associated values of        the first composition function.

In addition, there is provided a decoder comprising blending means, saidblending means comprising:

-   -   luminance generation means for generating a luminance component        of the blended object from the corresponding luminance        components of the first and second objects and from a first        composition function, and    -   chrominance generation means for a generating chrominance        component of the blended object from the corresponding        chrominance components of the first and second object and from a        second composition function, the second composition function        depending on a set of associated values of the first composition        function.

As we will see in detail further on, using a set of associated values ofthe first composition function for the second composition functionpermits to have a better precision for the chrominance component of anobject.

According to a first non-limited embodiment, the first compositionfunction depends on a shape component.

In a non-limited variant of this first non-limited embodiment, achrominance value is associated with 4 luminance values and 4 shapevalues, the second composition function being an ‘OR’ function betweenthe 4 associated shape values.

According to a second non-limited embodiment, the first compositionfunction is based on a transparency component.

In a non-limited variant of this second non-limited embodiment achrominance value is associated with 4 luminance values and 4transparency values, the second composition function being an average ofthe 4 transparency values.

BRIEF DESCRIPTION OF THE DRAWINGS

Additional objects, features and advantages of the invention will becomeapparent upon reading the following detailed description and uponreference to the accompanying drawings in which:

FIG. 1 illustrates a distribution of the chrominance and luminancecomponents in the case of the 4:2:0 format,

FIGS. 2 a, 2 b and 2 c illustrate a result of a chrominance computationof a blended object of a scene according to the prior art, and

FIG. 3 is a block diagram of a decoder.

DETAILED DESCRIPTION OF THE INVENTION

The present invention relates to a method for composing a scene, saidscene containing a plurality of objects, an object comprisingchrominance and luminance components, a chrominance value beingassociated with a set of at least two luminance values.

In the following description, the invention is depicted in the case of avideo format 4:2:0, i.e. one chrominance U or V value corresponds tofour luminance Y values, the chrominance being sub-sampled in thehorizontal and vertical directions, but it will be apparent to a personskilled in the art that the invention is applicable to any video formatin which the chrominance component is sub-sampled compared to theluminance component.

In the following description, well-known functions or constructions by aperson skilled in the art are not described in detail since they wouldobscure the invention in unnecessary detail.

FIG. 3 represents a general overview of a decoder structure. The samedecoding scheme is applied when decoding all the objects of a givensession.

The decoder comprises a demultiplexer DEMUX (31), a shape decoder SHDEC(32), a motion decoder MDEC (33) and a texture decoder YUVDEC (34) fordecoding the luminance and chrominance components. The reconstructedobject is obtained by the right combination of the shape, texture andmotion information thanks to a motion compensation circuit MC (35), areconstruction circuit (36) and a picture memory MEM (37), according toa principle known to a person skilled in the art.

After the decoding of the different objects of a given session, aninverse affine transform, such as rotation, a zoom or a reduction forexample, is applied on each object of a scene. To this end, a well-knowntechnique called <<backward mapping>> is used. The inverse affinetransform is applied on the coordinates (x,y) of each pixel of saidimage. The luminance, chrominance and transparency components of a pixelof the current object are then obtained by an interpolation, a bilinearinterpolation for example, between four pixels that are surrounding theinverse transformed pixel having coordinates (X,Y).

After the inverse affine transform, the blending step also calledcomposition is performed. The objects are recursively blended in apredetermined order.

In this step, a transparency value is associated to each object of thescene. Practically, it permits to see which object is behind or in frontof another object. Each object has its own luminance Y, chrominance Uand V, and transparency alpha values. Blending is done sequentially.Usually, the objects are composed two by two beginning by those on thebackward of a scene. For example, if a first object N is over-layeredover a second object M to generate a blended object P. The YUV pixelvalues of the blended object P depend on those of the objects M and Nand on the transparency value of the current object N. Hence:Py=[(255−Nalpha_(—) y)*My+Nalpha_(—) y*Ny]/255Puv=[(255−Nalpha_(—) uv)*Muv+Nalpha_(—) uv*Nuv]/255where:

-   -   Py and Puv are the luminance Y and chrominance UV components,        respectively, of the blended object P,    -   My and Muv are the luminance Y and chrominance UV components,        respectively, of the second object M, which is for example the        background of the scene, and which has been computed in a        preceding blending step and memorized,    -   Ny, Nuv are the luminance Y and chrominance UV components,        respectively, of the first object N being processed,    -   Nalpha_y and Nalpha_uv are the transparency components applied        to the pixels of an object N in order to obtained the Py and Puv        components, respectively,    -   the luminance value is comprised between 0 and 255.

A transparency component Nalpha is the product of three terms:

-   -   a global alpha value, which is applied to all the pixels of an        object,    -   some gray-alpha values, one value being associated with one        pixel,    -   some binary shape values, one value being associated with one        pixel.

Note that the transparency and the shape components have the sameresolution as the luminance.

More particularly, the luminance component of a blended object P isgenerated from the corresponding luminance components of the first andsecond objects N and M and from a first composition function. Thechrominance component of the blended object P is generated from thecorresponding chrominance components of the first and second objects Nand M and from a second composition function.

Thus, the transparency Nalpha_y of the luminance component is functionof the first composition function, and the transparency Nalpha_uv of thechrominance component is function of the second composition function.Advantageously, the second composition function depends on a set ofassociated values of the first composition function. It permits to bemore precise in the computation of the chrominance component Puv.

According to a first non-limited embodiment of the invention, the firstcomposition function is based on a binary shape component and thetransparency component Nalpha_uv is the product of global-alpha value,gray-alpha value and a shape value Shape_uv(x,y).

In this case, the second composition function is an OR of the shape ofthe corresponding four pixels such that:

Shape_uv(x,y)=Shape_y(2x−1,2y−1) OR Shape_y(2x,2y−1) OR Shape_y(2x−1,2y)OR

Shape_y(2x,2y) if pixels are numbered starting from 1.

For example for the current pixel whose coordinates are (x,y)=(2,1), thesecond composition function is:

Shape_uv(2,1)=shape_y(3,1) OR shape_y(4,1) OR shape_y(3,2) ORshape_y(4,2),

the different shapes values Shape_y( ) taking the values 0 or 1. Hence,the chrominance value is taken into account if at least one luminancevalue is taken into account, i.e. belongs to the object N.

With this second composition function, the resulting blended object “e”of FIG. 2 c has a much better quality, as it is the original “e”.

As for the gray-alpha value of the transparency Nalpha_uv, one can takeeither the current pixel gray-alpha value:

gray_alpha_uv(x,y)=gray_alpha_y(2x−1,2y−1),

or one can take the gray-alpha value of the four associated pixel, suchthat:

gray-alpha_uv(x,y)=[gray-alpha_y(2x−1,2y−1)+gray-alpha_y(2x,2y−1)+gray-alpha_y(2x−1,2y)+gray-alpha_y(2x,2y)]/4.

According to a second non-limited embodiment of the invention, the firstcomposition function is based on the transparency component.

In this case, the second composition function is an average of thetransparency of four associated pixels such that:

Nalpha_uv(x,y)=[Nalpha_y(2x−1,2y−1)+Nalpha_y(2x,2y−1)+Nalpha_y(2x−1,2y)+Nalpha_y(2x,2y)]/4

It is to be noted that the average can be computed in a different way,for example using weighting coefficient.

As the value of the transparency is comprised between 0 and 255, theresult gives a better precision than if only the shape is used as in thefirst embodiment.

It is to be understood that the present invention is not limited to theaforementioned embodiments and variations and modifications may be madewithout departing from the spirit and scope of the invention as definedin the appended claims. In the respect, the following closing remarksare made.

It is to be understood that the present invention is not limited to theaforementioned video application on mobile phone. For example it can beapplied on a television whenever there are some text message and video.

It is to be understood that the method according to the presentinvention is not limited to the aforementioned implementation.

There are numerous ways of implementing functions of the methodaccording to the invention by means of items of hardware provided that asingle item of hardware may carry out several functions. It does notexclude the possibility that an assembly of items of hardware may carryout a function, thus forming a single function without modifying themethod for composing a scene in accordance with the invention.

The hardware item can be implemented in several manners, such as bymeans of wired electronic circuits or by means of an integrated circuit.The integrated circuit can be contained in a computer or in an encoder.In the second case, the decoder may comprise a luminance generationmeans for generating a luminance component of the blended object fromthe corresponding luminance components of the first and second objectsand from a first composition function, and chrominance generation meansfor a generating chrominance component of the blended object from thecorresponding chrominance components of the first and second object andfrom a second composition function, the second composition functiondepending on a set of values of the first composition function, asdescribed previously, said means being hardware items as above stated.

The integrated circuit comprises a set of instructions. Thus, said setof instructions contained, for example, in a computer programming memoryor in a decoder memory may cause the computer or the decoder to carryout the different steps of the composing method.

The set of instructions may be loaded into the programming memory byreading a tangible data carrier such as, for example, a disk.

Any reference sign in the following claims should not be construed aslimiting the claim. It will be obvious that the use of the verb “tocomprise” and its conjugations do not exclude the presence of any othersteps or elements besides those defined in any claim. The article “a” or“an” preceding an element or step does not exclude the presence of aplurality of such elements or steps.

1. A method of using a decoder to blend a first object with a secondobject, thereby resulting in a blended object, each object comprisingchrominance and luminance components, a chrominance value beingassociated with a set of at least two luminance values, wherein saidmethod comprises: using the decoder to generate the luminance componentof the blended object from the corresponding luminance components of thefirst object and the second object, and from a first compositionfunction, the first composition function based upon at least atransparency component Nalpha y of the luminance component of the firstobject; and using the decoder to generate the chrominance component ofthe blended object from the corresponding chrominance components of thefirst object and the second object, and from a second compositionfunction, the second composition function depending on a set ofassociated values of the first composition function and based upon atleast a transparency component Nalpha uv of the chrominance component ofthe first object.
 2. The method of claim 1, wherein the chrominancevalue is associated with four luminance values and four transparencyvalues, the second composition function being an average of the fourtransparency values.
 3. The method of claim 1, wherein the firstcomposition function depends on a shape component.
 4. The method ofclaim 3, wherein the chrominance value is associated with four luminancevalues and four shape values, the second composition function being an‘OR’ function between the four associated shape values.
 5. A decoderthat blends a first object with a second object, thereby producing ablended object, each object comprising chrominance and luminancecomponents, a chrominance value being associated with a set of at leasttwo luminance values and at least one transparency value, said decodercomprising: luminance generation means for generating a luminancecomponent of the blended object from the corresponding luminancecomponents of the first object and the second object, and from a firstcomposition function, the first composition function based upon at leasta transparency component Nalpha y for the luminance component of thefirst object, and chrominance generation means for generating thechrominance component of the blended object from the correspondingchrominance components of the first object and the second object, andfrom a second composition function, the second composition functiondepending on a set of associated values of the first compositionfunction and based upon at least a transparency component Nalpha uv forthe chrominance component of the first obect.
 6. The decoder of claim 5,wherein each transparency component is a product of a global alphavalue, at least one gray alpha value, and at least one binary shapevalue.
 7. A decoder comprising: a demultiplexer coupled to at least afirst object and a second object, the first object and the second objectcomprising chrominance and luminance components; a shape decoder coupledto the demultiplexer; a motion decoder coupled to the demultiplexer; atexture decoder coupled to the demultiplexer; a motion compensationcircuit coupled to the shape decoder and the motion decoder; areconstruction circuit, coupled to the shape decoder, the texturedecoder, and the motion decoder, the reconstruction circuit producing ablended object as an output signal; and a picture memory coupled to thereconstruction circuit that provides a feedback signal to the motioncompensation circuit, wherein: the luminance component of the blendedobject is generated from the corresponding luminance components of thefirst and second objects and from a first composition function, thefirst composition function based upon at least a transparency componentNalpha_y for the luminance component of the first object, and thechrominance component of the blended object is generated from thecorresponding chrominance components of the first object and the secondobject, and from a second composition function, the second compositionfunction depending on a set of associated values of the firstcomposition function and based on at least a transparency componentNalpha_uv for the chrominance component of the first object.
 8. Themethod of claim 1, wherein each transparency component is a product of aglobal alpha value, at least one gray alpha value, and at least onebinary shape value.
 9. The decoder of claim 7, wherein each transparencycomponent is a product of a global alpha value, at least one gray alphavalue, and at least one binary shape value.